Skip to content

fix(calltrace): don't drop concurrent first inserts of the same stack (PROF-16805) - #1035

Open
kaahos wants to merge 5 commits into
mainfrom
paul.fournillon/sneaky-amber-ibis
Open

kaahos wants to merge 5 commits into
mainfrom
paul.fournillon/sneaky-amber-ibis

Conversation

@kaahos

@kaahos kaahos commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:

Stops CallTraceHashTable::put() from dropping a trace when two threads insert the same new stack at once.

  • A slot whose key matches but whose trace is not published yet (nullptr or PREPARING) is now treated as in progress. The waiter spins until the trace is published, or until the key changes, then probes the slot again. The "preparation failed" drop branch is removed: every abandon path clears the key, so that state never means failure.
  • After it sees a published trace, the waiter re-reads the key. A slot that was abandoned and reclaimed by another stack no longer returns that stack's ID.
  • A zero-frame stack hashes to 0, the empty-key value. put() now drops it explicitly before probing. This keeps today's behaviour; without the guard it would wait on an empty slot forever.
  • NativeSocketSampler now records its sample under CriticalSection, like every other put() caller. Without it, a profiling signal landing while the socket hook is inside put() could re-enter put() with the same Java-only stack and wait forever on its own thread's claim.

Motivation:

PROF-16805. The key CAS and markPreparing() are two steps. A second thread that probed between them saw key == hash, trace == nullptr and returned DROPPED_TRACE_ID. It hits start-up and restart bursts, where many threads first record the same stack (seen as a <dropped> stack in GCGenerationsTest on #663).

Additional Notes:

  • Human review: mandatory. Lock-free, signal-reachable, perf-critical code; please check the memory ordering and the wait semantics.
  • Accepted cost: the hit path does one extra load of the key it has just read.
  • Wait hazard, same class as today: a waiter on an owner stalled between the key CAS and the publish spins until the owner resumes, as the PREPARING wait already does over the longer storeCallTrace() window.
  • Same-thread re-entry: every put() caller now holds CriticalSection, so a nested signal handler drops its sample instead of waiting on its own thread's slot. The socket path was the one caller without it; this also closes the older self-deadlock in the PREPARING window.
  • Accepted cost: a profiling signal that lands while a socket sample is being recorded on the same thread drops its sample, as it already does during malloc and JVMTI sampling.
  • Rejected: dropping after a bounded wait (a preempted owner still drops), and publishing PREPARING before the key (a protocol redesign).
  • Follow-ups, not in this PR: remove the now-redundant PREPARING sentinel; findCallTrace() can create a duplicate ID for an in-progress slot in an older table.
  • Pre-PR review: Sphinx (generalist, SAST, mutation scouts). No Critical/High findings. Two test findings were refuted with evidence. Two surviving mutations were killed by asserting exact wait counts (reviewed with /code-review medium).

How to test the change?:

New deterministic gtests (CallTraceHashTableWaitTest) use a UNIT_TEST-only wait hook to play the concurrent owner on the waiting thread: claimed and PREPARING slots, each published and abandoned, reclaim by another stack, spin-limit re-probe, and zero-frame drop. A stress test races 8 threads on the same unseen stacks. NativeSocketSamplerCriticalSectionTest checks that the socket sample is recorded inside CriticalSection and dropped when the thread is already in one (Linux-only).

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a security review (run the dd:platform-security-review skill, or file a request via the PSEC review form). bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-16805

@kaahos kaahos added the human_review:mandatory Human review required — see PR description label Nov 20, 2026
@kaahos

kaahos commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

@DataDog review

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

A profiling signal can interrupt an unguarded native socket sample and wait on the interrupted thread’s unpublished slot. The new null-trace wait can therefore hang an application thread when both samples produce the same stack.

Open Bits AI session

🤖 Bits Code Review · Commit f90a155

@dd-octo-sts

dd-octo-sts Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #44832026667 | Commit: 9522f04 | Duration: 3h 0m 17s (longest job)

❌ 1 of 108 test jobs failed

Status Overview

JDK glibc-aarch64/asan glibc-aarch64/debug glibc-aarch64/release glibc-aarch64/tsan glibc-amd64/asan glibc-amd64/debug glibc-amd64/release glibc-amd64/tsan musl-aarch64/debug musl-aarch64/release musl-amd64/debug musl-amd64/release
8 - - - - ✅ ✅ ✅ ✅ - - - -
8-ibm - - - - ✅ 🚫 ✅ ✅ - - - -
8-j9 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
8-librca - - - - - - - - ✅ ✅ ✅ ✅
8-orcl - - - - ✅ ✅ ❌ ✅ - - - -
11 - - - - ✅ ✅ ✅ ✅ - - - -
11-j9 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
11-librca - - - - - - - - ✅ ✅ ✅ ✅
17 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
17-graal ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
17-j9 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
17-librca - - - - - - - - ✅ ✅ ✅ ✅
21 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
21-graal ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
21-librca - - - - - - - - ✅ ✅ ✅ ✅
25 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
25-graal ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ - - - -
25-librca - - - - - - - - ✅ ✅ ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

⚠️ Flaky tests — failed, then passed on retry

Test Cells Message
MallocArenaWarningTest.statsEventsDescribeTheArenas__ glibc-8-orcl-release-amd64 org.opentest4j.AssertionFailedError: expected: <482869248> but was: <477626368>

These fail the build. Passing on a second run makes a test flaky, not passing. Fix it, or quarantine it against a ticket so the debt is tracked rather than forgotten.

Consider quarantining these — click for ready-made entries

A quarantined test still runs and still reports; its failures just stop
turning CI red. To quarantine one:

  1. Open a PROF ticket for the test, linking the failing job.
  2. Append the line below to ddprof-test/quarantine.txt, replacing
    PROF-XXXXX with the ticket number.
  3. Check the cells and reason columns — the proposal only knows what
    failed in this run, and a narrower cells glob keeps the same test
    gating everywhere it has not misbehaved.

CI fails once review_by passes, so an entry expires instead of piling up.

# test | ticket | added | review_by | cells | reason
com.datadoghq.profiler.nativemem.MallocArenaWarningTest.statsEventsDescribeTheArenas | PROF-XXXXX | 2026-11-21 | 2027-02-19 | glibc-8-orcl-release-amd64 | org.opentest4j.AssertionFailedError: expected: <482869248> but was: <477626368> (seen in: glibc-8-orcl-release-amd64)

Retried 1 of 101 cells.

Failed Jobs

Summary: Total: 108 | Passed: 106 | Failed: 1 | Cancelled: 1


Updated: 2026-11-21 01:53:57 UTC

@dd-octo-sts

dd-octo-sts Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 0e7a4a70

@datadog-official

This comment has been minimized.

@kaahos

kaahos commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

@DataDog review

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The socket sampler’s CriticalSection guard covers the same-thread signal re-entry hazard when waiting on unpublished trace slots. Source review found no actionable defect in publication ordering or abandoned-slot retries; runtime and sanitizer behavior remain unverified.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 0811dc8

@kaahos
kaahos marked this pull request as ready for review November 20, 2026 12:53
@kaahos
kaahos requested a review from a team as a code owner November 20, 2026 12:53
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-20T12:55:15.637413Z 0811dc8 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kaahos kaahos added test:release Run CI tests with release configuration test:asan Run CI tests with AddressSanitizer configuration test:tsan Run CI tests with ThreadSanitizer configuration test:fuzz test:reliability labels Nov 20, 2026

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Static review found the publication wait and key revalidation consistent with slot ownership. The socket sampling guard addresses same-thread signal re-entry that could otherwise wait on its own unfinished insertion.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 0811dc8

@jbachorik jbachorik left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sphinx Review

Automated review findings for this pull request are posted as inline comments below.

Reviewed commit: 0811dc8

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means
  • 👍 Useful: correct, and you're fixing it or will.
  • 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
  • 😕 Unclear: you can't tell what it means or why it matters.
  • 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

🧞 Sphinx — automated review

jbachorik
jbachorik previously approved these changes Nov 20, 2026

@jbachorik jbachorik left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks feasible

@dd-octo-sts

dd-octo-sts Bot commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Reliability & Chaos Results

✅ All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/155385473

@kaahos

kaahos commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

@DataDog review

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The publication key recheck protects against returning a reclaimed slot’s trace, and the socket sampler’s CriticalSection prevents same-thread signal re-entry into the wait path. The completed static review identified no reportable defect.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 0e7a4a7

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

human_review:mandatory Human review required — see PR description mergequeue-status: removed test:asan Run CI tests with AddressSanitizer configuration test:fuzz test:release Run CI tests with release configuration test:reliability test:tsan Run CI tests with ThreadSanitizer configuration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants